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Amendment to the Claims : 

This listing of claims replaces all prior versions, and 
listings, of claims in the application: 

Listing of Claims : 

1-24. (cancelled) 

25. (previously presented) The method of claim 29, wherein 
the surface comprises a two dimensional surface. 

26. (original) The method of claim 25, wherein placing the 
two dimensional surface comprises: 

casting a ray from an optical center, corresponding to 
the real-time video imagery information, to a bottom point of 
the moving region in an image plane in the three dimensional 
model; and 

determining a position, an orientation and a size of 
the two dimensional surface based on the ray, a ground plane in 
the three dimensional model, and the moving region. 

27-28 . (cancelled) 

29. (previously presented) A method comprising: 
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obtaining a three dimensional model of a three 
dimensional environment, the three dimensional model generated 
from range sensor. information representing a height field for 
the three dimensional environment; 

identifying in real time a region in motion with 
respect to a background image in real-time video imagery 
information from at least one image sensor having associated 
position and orientation information with respect to the three 
dimensional model, the background image comprising a single 
distribution background dynamically modeled from a time average 
of the real-time video imagery information; 

placing a surface that corresponds to the moving 
region in the three dimensional model; 

projecting the real-time video imagery information 
onto the three dimensional model, including the surface, based 
on the position and orientation information; and 

visualizing the three dimensional model with the 
projected real-time video imagery; 

wherein identifying a region in motion in real time 
comprises subtracting the background image from the real-time 
video imagery information, identifying a foreground object in 
the subtracted real-time video imagery information, validating 
the foreground object by correlation matching between identified 
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objects in neighboring image frames, and outputting the 
validated foreground object; 

wherein identifying a foreground object comprises 
identifying the foreground object in the subtracted real-time 
video imagery information using a histogram-based threshold and 
a noise filter; 

wherein identifying a region in motion in real time 
further comprises estimating the background image by modeling 
the background image as a temporal pixel average of five recent 
image frames in the real-time video imagery information. 

30. (previously presented) The method of claim 29, further 
comprising tracking the position and orientation information of 
the at least one image sensor in the environment with respect to 
the three dimensional model in real-time. 

31. (previously presented) The method of claim 30, wherein 
obtaining a three dimensional model of a three dimensional 
environment comprises generating the three dimensional model of 
the three dimensional environment. 

32. (cancelled) 
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33. (previously presented) The system of claim 37, wherein 
the surface comprises a two dimensional surface. 

34. (original.) The system of claim 33, wherein the object 
detection and tracking component places the two dimensional 
surface by performing operations comprising: 

casting a ray from an optical center, corresponding to 
the real-time video imagery information, to a bottom point of 
the moving region in an image plane in the three dimensional 
model ; and 

determining a position, an orientation and a size of 
the two dimensional surface based on the ray, a ground plane in 
the three dimensional model, and the moving region. 

35-36 . (cancelled) 

37. (previously presented) An augmented virtual 
environment system comprising:. 

an object detection and tracking component that 
identifies in real time a region in motion with respect to a 
background image in real-time video imagery information from at 
least one image sensor having associated position and 
orientation information with respect to a three dimensional 
model of a three dimensional environment, the three dimensional 
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model generated from range sensor information representing a 
height field for the three dimensional environment, the 
background image comprising a single distribution background 
dynamically modeled from a time average of the real-time video 
imagery information, and places a surface that corresponds to 
the moving region with respect to the three dimensional model; 

a dynamic fusion imagery projection component that 
projects the real-time video imagery information onto the three 
dimensional model, including the surface, based on the position 
and orientation information; and 

a visualization sub-system that visualizes the three 
dimensional model with the projected real-time video imagery; . 

wherein the object detection and tracking component 
identifies the moving region by performing operations comprising 
subtracting the background image from the real-time video 
imagery information, identifying a foreground object in the 
subtracted real-time video imagery information, validating the 
foreground object by correlation matching between identified 
objects in neighboring image frames, and outputting the 
validated foreground object; 

wherein identifying a foreground object comprises 
identifying the foreground object in the subtracted real-time 
video imagery information using a histogram-based threshold and 
a noise filter; and 
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wherein identifying a region in motion in real time 
further comprises estimating the background image by modeling 
the background image as a temporal pixel average of five recent 
image frames in the real-time video imagery information. 

38. (previously presented) The system of claim 37, further 
comprising a tracking sensor system that integrates visual 
input, global navigational satellite system receiver input, and 
inertial orientation sensor input to obtain the position and the 
orientation information associated with the at least one image 
sensor in real time in conjunction with the real-time video 
imagery. 

39. (previously presented) The system of claim 38, further 
comprising a model construction component that generates the 
three dimensional model of the three dimensional environment. 

40-49. (cancelled) 
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